[Contents] [Prev. Chapter] [Next Section] [Next Chapter] [Index] [Help]

3    Shared SCSI Bus Requirements and Configuration

TruCluster Production Server Software allows you to configure one or more available server environments (ASEs), depending on the number of systems in the cluster. TruCluster Available Server Software allows you to configure one ASE. An ASE uses shared SCSI buses, external storage shelves or RAID controllers, and supports disk mirroring, and fast file system recovery to provide high data availability and reliability.

This chapter introduces SCSI bus configuration concepts and describes requirements for the shared SCSI bus. In addition to using only the supported hardware described in the TruCluster Software Products Release Notes, adhering to the requirements described in this chapter will ensure that your ASE operates correctly.

This chapter discusses the following topics:

In some sections of this manual, a distinction is made between the different implementations of the Small Computer System Interconnect (SCSI), SCSI, Fast 10 (F10), and UltraSCSI. Most of the manual uses SCSI as the generic term, which could be any of the SCSI implementations.


[Contents] [Prev. Chapter] [Next Section] [Next Chapter] [Index] [Help]

3.1    Shared SCSI Bus Configuration Requirements

A shared SCSI bus must adhere to the following requirements:


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.2    Numbering SCSI Buses

All member systems must recognize the disks on a shared bus at the same device number. A device number is obtained from the logical bus number, which is defined in the system configuration file. If you connect a shared bus to SCSI controllers that have the same logical bus number on each system, the shared disks will have the same device number on each system.

Bus numbers are assigned to SCSI controllers during the kernel configuration process, and they are specified in the kernel configuration file. When you configure a kernel by running the doconfig program using the generic kernel, an algorithm is used to probe the SCSI controllers installed in the system. As the probe algorithm encounters the adapters, it assigns logical bus numbers to the SCSI controllers in sequence, starting with 0.

Because systems have different numbers of internal buses, in an available server environment (ASE), the ase_fix_config script is used to manually assign logical bus numbers and override the probe algorithm. If you choose to have the software installation procedure rebuild your kernel, it automatically runs the script and prompts you for information about bus numbers. You can also manually invoke the script with the following command:

 # /var/ase/sbin/ase_fix_config -c CONFIG-FILE

CONFIG-FILE is the name of the system's configuration file, which is stored in the /sys/conf directory.

Before you install your SCSI controllers, plan your bus configuration. DIGITAL recommends that you install SCSI controllers for the local buses in the lower bus slots in your system, leave some empty slots, and then install the SCSI controllers for the shared buses. If you use this method, you can install additional SCSI controllers for both the local buses and the shared buses without disrupting the shared bus numbering scheme. For example, if a maximum of eight local controllers will be installed in a system, install the first controller for a shared bus in slot 8. Then, install another controller for a shared bus in the next highest slot (7), and so on.

See the TruCluster Software Products Software Installation manual for more information about the ase_fix_config script. In addition, some SCSI controllers have two ports (or channels) so you can connect two shared SCSI buses to each controller. For example, a PMAZC TURBOchannel SCSI controller has ports A and B, and a KZMSA XMI-to-SCSI adapter has channels 0 and 1. Both ports do not have to be used, but any unused ports must be terminated.

If you are using dual-port SCSI controllers, a shared bus must be connected to the same port on each system. For example, a shared bus that is connected to port A (or channel 0) on one SCSI controller must be attached to port A (or channel 0) on all the other SCSI controllers.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3    SCSI Bus Performance

Before you set up a SCSI bus, it is important that you understand a number of issues that affect the viability of a bus and how the devices connected to it operate. Specifically, bus performance is influenced by the following factors:

The following sections describe these factors.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3.1    Transmission Methods

Two transmission methods can be used in a SCSI bus:

You cannot use the two transmission methods in the same physical bus. For example, a device with a differential SCSI interface must be connected to another device with a differential SCSI interface. If you want to connect devices that use different transmission methods, use a SCSI signal converter between the devices. See Section 3.6 for information about using SCSI signal converters.

UltraSCSI does not support the use of a signal converter such as the DWZZA or DWZZB for Available Server or Production Server Version 1.5. The DS-BA35X-DA personality module functions as a signal converter for the UltraSCSI BA356. It is the interface between the shared differential UltraSCSI bus and the BA356 internal single-ended SCSI bus.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3.2    Data Path

There are two possible data paths for SCSI devices:

Usually, but not always, single-ended devices are narrow, and differential devices are wide. The BA356 storage shelves are examples of single-ended, wide devices. If you want to connect devices that have different data paths, use a SCSI signal converter between the devices.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3.3    Bus Speed

Bus speeds vary depending upon the type of SCSI bus: SCSI, SCSI-2, fast, wide SCSI-2, or UltraSCSI and the data path width. The following sections discuss these bus speeds in more detail.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3.3.1    SCSI, SCSI-2, and Fast Wide SCSI-2

SCSI and SCSI-2 controllers operate in two data transfer speeds: standard or slow mode and fast bus speed. In slow mode, the narrow 8-bit SCSI bus achieves up to 5 MB/sec. Fast SCSI-2 bus speed (the fast synchronous transfer option) enables I/O devices to attain high peak-rate transfers in synchronous mode. Production Server supports only fast bus mode.

To set the bus speed on a SCSI controller, use either console commands or the Loadable Firmware Update (LFU) utility, depending on the type of SCSI controller.

Although fast SCSI bus speed (sometimes referred to as Fast 10) doubles the transmission rate to 10 MB/sec, it reduces the maximum cable length for each single-ended bus from 6 meters to 3 meters. Fast bus speed on a wide (16-bit) SCSI bus increases the maximum bus bandwidth to 20 MB/sec (referred to as Fast 20). See Section 3.5 for more information about bus length.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.3.3.2    UltraSCSI Bus Speed

Fast bus speed in an UltraSCSI bus increases the maximim bus bandwidth to 40 MB/sec.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.4    SCSI Bus Device Identification Numbers

On a shared SCSI bus, each SCSI device uses a device address and must have a unique SCSI ID (from 0 to 7). For example, each SCSI bus adapter and each disk in a single-ended storage shelf uses a device address.

SCSI bus adapters have a default SCSI ID that you can change by using console commands or utilities. For example, a KZPSA adapter has an initial SCSI ID of 7.

Use the following priority order to assign SCSI IDs to the SCSI bus adapters connected to a shared SCSI bus:

7-6-5-4-3-2-1-0
 

This order specifies that 7 is the highest priority, and 0 is the lowest priority. When assigning SCSI IDs, use the highest priority ID for member systems (starting at 7). Use lower priority IDs for disks.

The SCSI ID for a disk in a BA350 or BA356 storage shelf corresponds to its slot location. In addition, you can set the SCSI IDs for some types of disks in a BA350 or BA356 storage shelf by using switches in the rear of some StorageWorks building blocks (SBBs). The SCSI ID for a disk in a BA353 storage shelf is set by the device address switches on the rear of the box.

The HSZ10 controller uses only one SCSI ID. You set the SCSI ID for an HSZ10 controller using switches on the unit. You can configure the HSZ40 controller with one to four SCSI target addresses.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.5    SCSI Bus Length

There is a limit to the length of the cables in a shared SCSI bus. The total cable length for a physical bus or UltraSCSI bus segment is calculated from one terminated end to the other.

If you are using devices that have the same transmission method and data path (for example, wide differential), a shared bus will consist of only one physical bus. If you have devices with different transmission methods, you will have both single-ended and differential physical buses, each of which must be terminated only at both ends and must adhere to the rules on bus length.

Note

In an Available Server or Production Server configuration you always have single-ended SCSI bus segments since all of the storage shelves use a single-ended bus.

Table 3-1 describes the maximum cable length for a physical SCSI bus.

Table 3-1:  SCSI Bus Length

SCSI Bus Bus Speed Maximum Cable Length
Narrow, single-ended 5 MB/second 6 meters
Narrow, single-ended fast 10 MB/second 3 meters
Wide differential, fast 20 MB/second 25 meters
Differential UltraSCSI 40 MB/second 24 meters [Footnote 6]

The total length of a physical bus must include the amount of cable that is located inside each system and disk storage shelf. This length varies, depending on the device. For example, the length of cable inside a BA350, BA353, or BA356 storage shelf is approximately 1.0 meter.

Because of the cable length limit, you must plan your hardware configuration carefully, and ensure that each SCSI bus meets the cable limit guidelines. In general, you must place systems and storage shelves as close together as possible and choose the shortest possible cables for the shared bus.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.6    Using Signal Converters

A SCSI signal converter allows you to couple a differential bus segment to a single-ended bus segment, allowing the mixing of differential and single-ended devices on the same bus to isolate bus segments for maintenance purposes.

Each SCSI signal converter has a single-ended side with either a narrow or a wide data path, and a differential side with a wide data path.

Note

Some UltraSCSI documentation uses the UltraSCSI "bus expander" term when referring to the DWZZA, DWZZB, and UltraSCSI signal converters. Other UltraSCSI documentation refers to some UltraSCSI products as bus extender/converters.

For Available Server and Production Server Version 1.5 there are no supported UltraSCSI bus expanders (DWZZC).

In this manual, any device that converts a differential signal to a single-ended signal is referred to as a signal converter, except the DS-BA35X-DA personality module. However, because the DS-BA35X-DA personality module does convert a differential bus to a single-ended bus, it is discussed with the signal converters.

A SCSI signal converter is required in the following cases:


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.6.1    Types of SCSI Bus Signal Converters

Signal converters can be standalone units, StorageWorks building blocks (SBBs) that are installed in a storage shelf disk slot, or as with the UltraSCSI BA356, the SBB Shelf Differential I/O Module (pesonality module DS-BA35X-DA). You must use the signal converter or personality module that is appropriate for your hardware configuration.

For example, use a DWZZA-VA signal converter to connect a BA350 or BA353 (single-ended and narrow) storage shelf to a differential device, but use a DWZZB-VW signal converter to connect a non-UltraSCSI BA356 (single-ended and wide) storage shelf to a differential adapter. The DS-BA35X-DA personality module is used in an UltraSCSI BA356 to connect a differential adapter to the single-ended disks in the BA356.

Table 2-7 shows the types of supported SCSI signal converters.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.6.2    SCSI Signal Converter Removable Termination

The DWZZA and DWZZB signal converters have removable termination. The DS-BA35X-DA personality module has switch selectable termination. The following sections describe these signal converters in more detail.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.6.2.1    DWZZA and DWZZB Signal Converter Termination

Both the single-ended side and the differential side of each DWZZA and DWZZB signal converter has removable termination. To use a signal converter, you must remove the termination in the differential side and attach a trilink connector to this side. To remove the differential termination, remove the five 14-pin SIP resistors (located near the differential end of the signal converter). You can terminate the trilink connector to terminate the differential bus. If you detach the trilink connector from the signal converter, the shared SCSI bus is still terminated.

In most cases, you must keep the termination in the single-ended side to terminate the single-ended bus connecting the signal converter and the single-ended device. Verify that the termination is active. A DWZZA should have jumper J2 installed. Jumpers W1 and W2 should be installed in a DWZZB.

However, if you have a DWZZA-VA signal converter installed in a BA353 storage shelf, you must remove the termination from the single-ended side. To do this, remove jumper J2.

Figure 3-1 shows the status of internal termination for a standalone SCSI signal converter that has a trilink connector attached to the differential side.

Figure 3-1:  Standalone SCSI Signal Converter

Figure 3-2 shows the status of internal termination for an SBB SCSI signal converter that has a trilink connector attached to the differential side.

Figure 3-2:  SBB SCSI Signal Converter


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.6.2.2    DS-BA35X-DA Termination

The UltraSCSI BA356 shelf uses a 16-bit differential UltraSCSI personality module (DS-BA35X-DA) as the interface between the UltraSCSI differential bus and the UltraSCSI single-ended bus in the BA356.

The personality module controls termination for the external differential UltraSCSI bus segment, and for both ends of the internal UltraSCSI single-ended bus segment.

For normal cluster operation, the differential termination must be disabled since a trilink connector will be installed on personality module connector JA1, which allows the use of the BA356 in the middle of the bus or external termination for a BA356 on the end of the bus.

Switch pack 4 switches S4-1 and S4-2 are set to ON to disable the personality module differential termination. The switches have no effect on the BA356 internal, single-ended UltraSCSI bus termination.

Note

S4-3 and S4-4 have no function on the DS-BA35X-DA personality module.

Figure 3-3 shows the relative positions of the two DS-BA35X-DA switch packs.

Figure 3-3:  DS-BA35X-DA Personality Module Switches


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.7    SCSI Bus Termination

You must properly connect devices to a shared SCSI bus. In addition, you can terminate only the beginning and end of each bus segment (either single-ended or differential).

There are two rules for SCSI bus termination:

Note

Generally, when using tape loaders on a shared SCSI bus, cluster SCSI bus termination principles will not be adhered to. The SCSI bus is not terminated externally to the tape loader. Therefore, the tape loader must be on the end of the shared SCSI bus.

Whenever possible, connect devices to a shared bus so that they can be isolated from the bus. This allows you to disconnect devices from the bus for maintenance purposes without affecting bus termination and cluster operation. You also can set up a shared SCSI bus so that you can connect additional devices at a later time without affecting bus termination.

Most devices have internal termination or some other method of termination. For example, PMAZC, KZTSA, and KZPSA adapters, BA350 and BA356 storage shelves, SCSI signal converters, and the UltraSCSI BA356 personality module (DS-BA35X-DA) have internal termination, but BA353 and HSZ40 controllers have automatic termination. Depending on how you set up a shared bus, you may have to enable or disable device termination.

If you use a device's internal termination to terminate a shared bus, and you disconnect the bus cable from the device, the bus will not be terminated and cluster operation will be impaired. Therefore, you must use external termination, enabling you to detach the device without affecting the bus termination.

To be able to externally terminate a bus and connect and disconnect devices without affecting bus termination, remove the device termination and use Y cables or trilink connectors to connect a device to a shared SCSI bus.

By attaching a Y cable or trilink connector to an unterminated device, you can locate the device in the middle or at the end of the shared bus. If the device is at the end of a bus, attach a terminator to the Y cable or trilink connector to terminate the bus. If you disconnect the Y cable or trilink connector from the device, the shared bus is still terminated and the shared SCSI bus is still operable.

In addition, you can attach a Y cable or a trilink connector to a shared bus without connecting the Y cable or trilink connector to a device. If you do this, you can connect a device to the Y cable or trilink connector at a later time without affecting bus termination. This allows you to expand your configuration without shutting down the cluster.

Figure 3-4 shows a BN21V-0B Y cable, which you attach to a PMAZC TURBOchannel SCSI controller that has had its onboard termination removed.

Figure 3-4:  BN21V-0B Y Cable

Figure 3-5 shows a BN21W-0B Y cable, which you may attach to a KZTSA or KZPSA SCSI adapter that has had its onboard termination removed. You can use the BN21W-0B Y cable with an HSZ10, HSZ40, or HSZ50 controller. Use a trilink connector with a KZTSA instead of the Y cable.

Figure 3-5:  BN21W-0B Y Cable

Figure 3-6 shows an HD68 trilink connector (H885), which you may attach to a KZTSA or KZPSA adapter that has its onboard termination removed, an HSZ10, HSZ40, or HSZ50 controller, or the unterminated differential side of a SCSI signal converter.

Figure 3-6:  HD68 Trilink Connector (H885)

Note

If you connect a trilink connector to a SCSI bus adapter, you may block access to an adjacent PCI slot. If this occurs, use a Y cable instead of the trilink connector. This is the case with the KZPSA SCSI adapters on some AlphaServer systems.

Figure 3-7 shows a VHDCI trilink connector (UltraSCSI), which you may attach to an HSZ70 or UltraSCSI BA356 personality module that has the external SCSI bus termination disabled.

Figure 3-7:  VHDCI Trilink Connector (H8861-AA)

Figure 3-8 shows an Available Server shared SCSI bus that includes two DEC 3000 Model 500 systems with PMAZC TURBOchannel SCSI controllers installed, a BA350 storage shelf, and one single-ended bus. A BN21V-0B Y cable is attached to the unterminated A port in each PMAZC controller. (The unused B port is terminated with terminator part number 12-41667-01.) The BA350 storage shelf is in the middle of the bus, so its internal termination is removed. The shared bus is terminated by H8574-A or H8860-AA terminators attached to each BN21V-0B cable.

Figure 3-8:  Terminating a Single-Ended Bus with Y Cables

If this configuration is used and a Y cable is disconnected from a PMAZC controller, that system is not available. However, the shared SCSI bus is still operable because the bus termination is maintained, as shown in Figure 3-9.

Figure 3-9:  Disconnecting a Y Cable

If you have the same hardware as described in Figure 3-9, you can create an alternative hardware configuration by placing one of the systems in the middle of the shared bus and the storage shelf at the end of the bus. In this case, the BA350 storage shelf internal termination is used to terminate the bus, as shown in Figure 3-10.

Figure 3-10:  Terminating a Single-Ended Bus at the Storage Shelf

However, if you have the configuration shown in Figure 3-10 and you disconnect the cable from the BA350 storage shelf, the single-ended shared bus is not terminated and ASE operation is impaired. To be able to isolate a single-ended storage shelf from a shared SCSI bus, you must connect the shelf to the single-ended side of a SCSI signal converter and attach a trilink connector to the differential side. Using this configuration, you can disconnect both the storage shelf and the signal converter from the shared bus without affecting ASE operation.

Figure 3-11 shows a hardware configuration that uses signal converters. The entire shared bus consists of three single-ended buses and one differential bus.

Figure 3-11:  Using SCSI Signal Converters on a Bus

In Figure 3-11, there is a single-ended bus between each single-ended device (KZMSA adapter or BA350 storage shelf) and the single-ended side of a DWZZA-AA signal converter. The single-ended bus is terminated by the internal termination in the device and in the signal converter, as shown in Figure 3-12. If a cable is disconnected from the single-ended device, the single-ended bus is not terminated; however, the differential bus is not affected if you have turned off the SCSI signal converter.

Figure 3-12:  Terminating a Single-Ended Bus

In Figure 3-11, the differential bus is connected to the trilink connectors attached to the unterminated differential side of each DWZZA-AA signal converter. The differential bus is terminated by the terminators attached to the trilink connectors at the ends of the bus, as shown in Figure 3-13. If a trilink connector is disconnected from a DWZZA-AA, the differential bus is still terminated.

Figure 3-13:  Terminating a Differential Bus


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.8    Connecting Devices on a Shared SCSI Bus

There are a number of requirements for connecting devices on a SCSI bus. The requirements are:

All the member systems and at least one storage shelf are connected to each shared SCSI bus. You can directly connect two devices only if they have the same transmission method (single-ended or differential) and data path (narrow or wide). The SCSI cable used to connect devices depends on the transmission mode and data path of the devices.

The following sections describe how to connect devices on a shared SCSI bus. After you understand the SCSI bus connections that you need for your configuration, you can prepare the systems and storage shelves.


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.8.1    Connecting Single-Ended Devices

Single-ended devices usually have a narrow data path. However, the BA356 storage shelf is an example of a single-ended device that has a wide data path. You cannot directly connect devices with different data paths; instead, you must use two SCSI signal converters to connect the devices. See Section 3.8.3 for information about connecting narrow and wide devices.

Figure 3-14 shows the cable required for a bus connection between two single-ended, narrow devices, which include:

Figure 3-14:  Bus Connection Between Single-Ended, Narrow Devices

Figure 3-15 shows the cable required for a bus connection between two single-ended, wide devices, which include:

Figure 3-15:  Bus Connection Between Single-Ended, Wide Devices


[Contents] [Prev. Chapter] [Prev. Section] [Next Section] [Next Chapter] [Index] [Help]

3.8.2    Connecting Differential Devices

Differential devices always have a wide data path. Figure 3-16 shows the cable required for a bus connection between two differential, wide devices, which include:

Figure 3-16:  Bus Connection Between Differential Devices


[Contents] [Prev. Chapter] [Prev. Section] [Next Chapter] [Index] [Help]

3.8.3    Connecting Single-Ended Devices to Differential Devices

Use a SCSI signal converter to connect devices with different transmission modes or data paths. See Section 3.6 for detailed information about using SCSI signal converters.

Figure 3-17 shows how to connect a device with a single-ended and narrow SCSI interface (for example, a BA350 or BA353 storage shelf) to a device with a differential and wide SCSI interface (for example, a KZPSA adapter) by using a DWZZA-VA signal converter installed in a disk slot.

Figure 3-17:  Bus Connection Between a Single-Ended, Narrow Device and a Differential Device

If instead of using a DWZZA-VA you use a standalone DWZZA-AA signal converter, connect the DWZZA-AA to the single-ended, narrow device using a BN21R or BN23G cable.

Figure 3-18 shows how to connect a device with a single-ended and wide SCSI interface (for example, a BA356 storage shelf) to a device with a differential and wide SCSI interface by using a DWZZB-VW signal converter, which is installed in a disk slot.

Figure 3-18:  Bus Connection Between a Single-Ended, Wide Device and a Differential Device

If instead of using a DWZZB-VW you use a standalone DWZZB-AA signal converter, connect the DWZZB-AA to the single-ended, wide device using a BN21K or BN21L cable.

To connect two devices with single-ended SCSI interfaces, connect each device to a SCSI signal converter, and then connect the differential sides of the signal converters by using a BN21K or BN21L cable.


[Contents] [Prev. Chapter] [Prev. Section] [Next Chapter] [Index] [Help]